Nonlinear Two-Time-Scale Stochastic Approximation Convergence and Finite-Time Performance

Two-time-scale stochastic approximation, a generalized version of the popular has found broad applications in many areas including control, optimization, and machine learning. Despite its popularity, theoretical guarantees this method, especially finite-time performance, are mostly achieved for linear case while results nonlinear counterpart very sparse. Motivated by classic control theory singularly perturbed systems, we study paper asymptotic convergence analysis two-time-scale approximation. Under some fairly standard assumptions, provide formula that explicitly characterizes rate main iterates to desired solutions. In particular, show mean square error generated method convergences zero at $\mathcal {O}(1/k^{2/3})$ , where notation="LaTeX">$k$ is number iterations. The key idea our properly choose two step sizes characterize coupling between fast slow-time-scale iterates.